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Robust State Space Filtering under Incremental 
Model Perturbations subject to a Relative 

Entropy Tolerance 

Bernard C. Levy and Ramine Nikoukhah 

Abstract 

This paper considers robust filtering for a nominal Gaussian state-space model, when a relative 
entropy tolerance is applied to each time increment of a dynamical model. The problem is formulated 
as a dynamic minimax game where the maximizer adopts a myopic strategy. This game is shown to 
admit a saddle point whose structure is characterized by applying and extending results presented earlier 
in |[T| for static least-squares estimation. The resulting minimax filter takes the form of a risk-sensitive 
filter with a time varying risk sensitivity parameter, which depends on the tolerance bound applied 
to the model dynamics and observations at the corresponding time index. The least-favorable model 
is constructed and used to evaluate the performance of alternative filters. Simulations comparing the 
proposed risk-sensitive filter to a standard Kalman filter show a significant performance advantage when 
applied to the least-favorable model, and only a small performance loss for the nominal model. 

Index Terms 

commitment, dynamic minimax game, least-favorable model, myopic strategy, relative entropy, risk- 
sensitive filtering, robust filtering. 

I. Introduction 

Soon after the introduction of Wiener and Kalman filters, it was recognized that these filters 
were vulnerable to modelling errors, in the form of either parasitic signals or perturbations of the 
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system dynamics. Various approaches were proposed over the last 35 years to construct filters 
with a guaranteed level of immunity to modelling uncertainties. Drawing from the framework 
developed by Huber for robust statistics [2J, Kassam, Poor and their collaborators proposed an 
approach Il3l-|l5l where the optimum filter is selected by solving a minimax problem. In this 
approach, the set of possible system models is described by a neighborhood centered about the 
nominal model, and two players affront each other. One player (say, nature) selects the least- 
favorable model in the allowable neighborhood and the other player designs the optimum filter 
for the least-favorable model. While minimax filtering is conceptually simple, its implementa- 
tion can be very difficult, since it depends on the specification of the allowable neighborhood 
and of the loss function to be minimized. After some early success in the design of Wiener 
filters for neighborhoods specified by e-contamination models or power spectral bands, progress 
stalled gradually and researchers started looking in different directions to develop robust filters. 
The 1980s saw the development of an entirely different class of robust filters based on the 
minimization of risk-sensitive and H°° performance criteria [l6ll- [|T0ll . This approach seeks to 
avoid large errors, even if these errors are unlikely based on the nominal model. For example, 
risk-sensitive filters replace the standard quadratic loss function of least-squares filtering by an 
exponential quadratic function, which of course penalizes severely large errors. However, errors 
in the model dynamics are not introduced explicitly in H°° and risk-sensitive filtering, and the 
growing awareness of the importance of such errors prompted a number of researchers in the 
early 2000s [|TT1l - [fT3l to revive the minimax filtering viewpoint, but in a context where modelling 
errors are described in terms of norms for state-space dynamics perturbations. The present paper, 
which is a continuation of |fT]|, can be viewed as part of a larger effort initiated by Hansen and 
Sargent [|T4ll - [fT6ll and other researchers ifTTl . [fTSll which is aimed at reinterpreting risk-sensitive 
filtering from a minimax viewpoint. In this context, modelling uncertainties are described by 
specifying a tolerance for the relative entropy between the actual system and the nominal model. 
To a fixed tolerance level describing the modeller's confidence in the nominal model corresponds 
a ball of possible models for which it is then possible to apply the minimax filtering approach 
proposed by Kassam and Poor. 

The minimax formulation of robust filtering based on a relative entropy constraint has several 
attractive features. First, relative entropy is a natural measure of model mismatch which is 
commonly used by statisticans for fitting statistical models by using techniques such as the 
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expectation maximization iteration fT9]|- More fundamentally, it was shown by Chentsov (201 
and by Amari [|2T| that manifolds of statistical models can be endowed with a non-Riemannian 
differential geometric structure involving two dual connections associated to the relative entropy 
and the reverse relative entropy. In addition to this strong theoretical justification, it turns out that 
minimax Wiener and Kalman filtering problems with a relative entropy constraint admit solutions 
[El, [[T4l|. ifTSl . ifTSl in the form of risk-sensitive filters, thus providing a new interpretation for 
such filters. The main difference between earlier works and the present paper is that, instead 
of placing a single relative entropy constraint on the entire system model, we apply a separate 
constraint to each time increment of the model. This approach, which is closer to the one 
advocated in [[T2l. lfT3l is based on the following consideration. When a single relative entropy 
constraint is placed on the complete system model, the maximizing player has the opportunity 
to allocate almost all of its mismatch budget to a single element of the model most susceptible 
to uncertainties. But this strategy may lead to overly pessimistic conclusions, since in practice 
modellers allocate the same level of effort to modelling each time component of the system. 
Thus it probably makes better sense to specify a fixed uncertainty tolerance for each model 
increment, instead of a single bound for the overall model. 

The analysis presented relies in part on applying and extending static least-squares estimation 
results derived in HJ for nominal Gaussian models. These results are reviewed in Section UIl In 
Section Uni the robust state-space filtering problem with an incremental relative entropy constraint 
is formulated as a dynamic game where the maximizer adopts a myopic strategy whose goal is 
to maximize the mean-square estimation error at the current time only. The existence of a saddle 
point is established in Section |lVl where the least-favorable model specifying the saddle point 
is characterized by extending a Lemma of |[T1 to the dynamic case. A careful examination of the 
least-favorable model structure allows the transformation of the dynamic estimation game into 
an equivalent static one, to which the result of become applicable. The robust filter that we 
obtain is a risk-sensitive filter, but with a time-varying risk-sensitivity parameter, representing 
the inverse of the Lagrange multiplier associated to the model component constraint for the 
corresponding time period. The least-favorable state-space model for which the robust filter is 
optimal is constructed in Section |Vl This model extends to the finite-horizon time- varying case a 
model derived asymptotically in [14] for the case of constant systems. The least-favorable state- 
space model allows performance evaluation studies comparing the performance of the minimax 



September 27, 2011 



DRAFT 



4 



filter with that of other fikers, such as the standard Kalman fiker. Simulations are presented 
in Section |VI] which illustrate the dependence of the filter performance on the relative entropy 
tolerance applied to each model time component. The robust filter is compared to the ordinary 
Kalman filter by examining their respective performances for both the nominal and least-favorable 
systems. Finally, some conclusions are presented in Section IVIII 

II. Robust Static Estimation 

We start by reviewing a robust static estimation result derived in [[U, since its extension to 
the dynamic case is the basis for the robust filtering scheme we propose. Let 



(2.1) 



X 

y 

be a random vector of ]R"+p, where x G M" is a vector to be estimated, and y & W h an 
observed vector. The nominal and actual probability densities of z are denoted respectively as 
f{z) and f{z). The deviation of / from / is measured by the relative entropy (KuUback-Leibler 
divergence) 

D{fJ)= I ~f{z)\r^(M\dz. (2.2) 



The relative entropy is not a distance, since it is not symmetric and does not satisfy the triangle 
inequality. However it has the property that D(f,f) > with equality if and only if / = /. 
Furthermore, since the function 9{£) = i\n{i) is convex for < £ < oo, D{f,f) is a convex 
function of /. For a fixed tolerance c > 0, if J-" denotes the class of probability densities over 
W+P, this ensures that the "ball" 

B = {feJ^: D{f,f)<c} (2.3) 

of densities / located within a divergence tolerance c of the nominal density / is a closed convex 
set. B represents the set of all possible true densities of random vector z consistent with the 
allowed mismodelling tolerance. 

Throughout this paper we shall adopt a minimax viewpoint of robustness similar to f2}\, 
[[141. where whenever we seek to design an estimator minimizing an appropriately selected 
loss function, a hostile player, say "nature," conspires to select the worst possible model in 
the allowed set, here B, for the performance index to be minimized. This approach is rather 



September 27, 2011 



DRAFT 



5 



conservative, and the performance of estimators in the presence of modelling uncertainties could 
be evaluated differently, for example by averaging the performance index over the entire ball 
B of possible models. However this averaging operation is computationally demanding, as it 
requires a Monte Carlo simulation, and typically does not yield analytically tractable results. It 
is also worth pointing out that the degree of conservativeness resulting from the selection of 
a minimax estimator can be controlled by appropriate selection of the tolerance parameter c, 
which ensures that an adequate balance between performance and robustness is reached. 
In this paper, we shall use the mean-square error (scaled by 1/2) 

1 21 



^ E[\\x - giy)\\' 



\x-g{y)\\'f{z)dz 



(2.4) 



to evaluate the performance of an estimator x = g(y) of x based on observation y. In (|2.4I) . if 
V denotes a vector of M" with entries Vi, 



n 



1/2 



denotes the usual Euclidean vector norm. Let Q denote the class of estimators such that E[x^] 
is finite for all f E B. Then the optimal robust estimator solves the minimax problem 



minmax J(/, g) . 



(2.5) 



Since the functional J(/, g) is quadratic in g, and thus convex, and linear in /, and thus concave, 
a saddle -point (/q, go) of minimax problem (12.51 ) exists, so that 



J{Lg,)<J{k9o)<JCh 



(2.6) 



However, characterizing precisely this saddle-point is difficult, except when the nominal density 
is Gaussian, i.e. 

/(^)~iV(m„ir,), (2.7) 

where in conformity with partition (|2.1I) of z, the mean vector and covariance matrix Kz 
admit the partitions 



K K 

^^yx ^^y 
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Then it was shown in Theorem 1 of [[U (see also lfT4l Sec. 7.3] for an equivalent result derived 
from a stochastic game theory perspective) that the saddle point of minimax poblem (|2.5I) admits 
the following structure. 



Theorem 1: If / admits the Gaussian distribution (12. 7|) . the least-favorable density /o is also 
Gaussian with distribution 

fo^N{m,,K,), (2.8) 



where the covariance matrix 



K K 



(2.9) 



is obtained by perturbing only the covariance matrix of x, leaving the cross- and co-variance 
matrices K^y and Ky unchanged. Accordingly, the robust estimator 



X = go{y) = mx + KxyK \y 



(2.10) 



coincides with the usual least-squares estimator for nominal density /. The perturbed covariance 
matrix K,. can be evaluated as follows. Let 



K - K K~^K 

^^x ^^xy^^y ^^yx 



P 

P = Kx-KxyKy'Kyx (2.11) 
denote the nominal and least-favorable error covariance matrices of x given y. Then 

= P-' - \-'ln , (2.12) 

where A denotes the Lagrange multiplier corresponding to constraint D{f,f) < c. Note that to 
ensure that P is a positive definite matrix, we must have A > r(P), where r(P) denotes the 
spectral radius (the largest eigenvalue) of P. 

To explain precisely how A is selected to ensure that the Karush-Kuhn-Tucker (KKT) condition 

A(c-D(/o,/)) = (2.13) 

holds, observe first that for two Gaussian densities / ~ N{mz,Kz) and / ~ N{mz, K^), the 
relative entropy can be expressed as [[22| 

1 



D{f, /) = - ||Am,||5,-i + tr {K;'K, - - Indet (i^^'^.) 



(2.14) 
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rriz and ||f||i^-i 



densities specified by (|2.7I) and (I2.8I) - (I2.9I) . we have Am 

= 

K. = 



K ^vY^"^. Then for the nominal and least-favorable 
and 



In 


Go 




P 







In 








/p _ 







ir, _ 






I,_ 


In 


Go 




P 







In 








h _ 







Ky _ 









where Gq = K^yKy ^ denotes the gain matrix of the optimal estimator (|2.10|) . Then after simple 
algebraic manipulations, we find 

1 



D{foJ) = - tr(PP-i-J„)-lndet(PP-i^ 



Substituting (|2.12l) gives 



A 



7(A) = D(/o,/) 



1 



tr ((/„, - X-'P)-' - In) + lndet(/„ - X'^P) 



(2.15) 



(2.16) 



where 7(A) is differentiable over (r(P), 00). By using the matrix differentiation identities ||23l 
Chap. 8] 



^rlndetMfA) 
dX 



tr [M- 



.idM 

'dX^ 
^dM 

ItX 



for a square invertible matrix function M(A), we find 

(i7 



dX 



-tr [(/ - A-^P)-^A-=^P2(J - X-^P)-^] < 
so that 7(A) is monotone decreasing over (r(P), 00). Since 

lim 7(A) = 



(2.17) 



lim 7(A) = +00 

A^r(P) 



A^oo 



this ensures that for an arbitrary tolerance c > 0, there exists a unique A > r(P) such that 
7(A) = c. 

For the case where the nominal density / is non-Gaussian, some results characterizing the 
solution of the minimax problem (12.51) were described recently in [24]. In addition, it is worth 
noting that it is assumed in Theorem 1 that the whole density f{z) = f{x,y) is subject to 



September 27, 2011 



DRAFT 



8 



uncertainties. But this assumption does not fit all situations. Consider for example a mutiple- 
input multiple-output (MIMO) least-squares equalization problem for a flat channel described 
by the nominal linear model 

y = Cx + v (2.18) 

where x denotes the transmitted data, C is the channel matrix and v ~ A^(0, R) represents the 
channel noise, which is assumed independent of x. Since the transmitted data x is under the 
control of the designer, its probability distribution f{x) is known exactly and it is not realistic 
to assume that it is affected by modelling uncertainties. Thus if f{y\x) ~ N(Cx, R) denotes the 
nominal conditional distribution specified by (12.181) . the actual density of z can be represented 
as 

f{z) = f{y\x)f{x) , 

where f{y\x) represents the true channel model, and where the data density f{x) is not perturbed. 
This constraint changes the structure of the minimax problem (12.51) . and a solution to this modified 
problem is presented in L24J and [|25l . 

in. Robust Filtering Viewed as a Dynamic Game 

We consider a robust state-space filtering problem for processes described by a nominal Gauss- 
Markov state-space model of the form 

xt+i = AtXt + BtVt (3.1) 

yt = CtXt + DtVt , (3.2) 

where Vt G M*" is a WGN with unit variance, i.e., 

E[vtv'^,] = I^Sit - s) , (3.3) 

where 

f 1 r = 

5{r) = 

1^ r ^ 

denotes the Kronecker delta function. The noise vt is assumed to be independent of the initial 
state, whose nominal distribution is given by 

/o(xo)~iV(mo,Po). (3.4) 
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Let 



Zt 



A 



yt 



The model (I3.1I) - (I3.3I) can be viewed as specifying the nominal conditional density 

(i)t{zt\xt) ~ N[ 



At 




Bt 




Xt, 




Ct 




Dt 



BT Df 



(3.5) 



of Zt given Xt. We assume that the noise Vt affects all components of the dynamics (13.11) and 
observations (13.21) . so that the covariance matrix 



K 



zt\xt 



Bt 
Dt 



BJ Df 



(3.6) 



is positive definite. To interpret this assumption, observe that in general, state-space models of 
the form (|3.1I) - (|3.2I) are formed by a mixture of noisy and deterministic linear relations (see 
for example the decomposition employed in [|26ll ). This means that the resulting conditional 
densities are concentrated on lower-dimensional subspaces of R"+p. As soon as these subspaces 
are slightly perturbed, it is possible to discriminate perfectly between the nominal and perturbed 
models, i.e., the relative entropy of the two models is infinite (the corresponding probability 
measures are not absolutely continuous with respect to each other). Accordingly, when the 
relative entropy is used to measure the proximity of statistical models, all deterministic relations 
between dynamic variables or observations are interpreted as immune from uncertainty, and 
only relations where noise is already present can be perturbed. Since this limitation is rather 
unsatisfactory, it is convenient to assume, like earlier robust filtering studies [[T2|. ifTTl . ifTSl . that 
the noise Vt affects all components of the dynamics and observations, possibly with a very small 
variance for relations which are viewed as almost certain. 
In this case, since the matrix 

A Bt 

Tt = 

Dt 

has full row rank, we can assume without loss of generality that Tt is square and invertible, so 
that m = n + p. Otherwise, if m > n + p, we can find an m x m orthonormal matrix Ut which 
compresses the columns of Tt, so that 



TtUt 



Tt 
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where r+ is invertible. Then if we denote 



Vt 



we have 



VtVt = TtVt 



where Vt is a zero-mean WGN of dimension n + p with unit covariance matrix. 
Over a finite interval < t < T, the joint nominal probability density of 



Xt+1 



Xo 



Xt+1 



and Yt 



yt 



Vt 



can be expressed as 



/(Xt+1, It) = h{xo)Y\(t)t{,zt\xt) 



(3.7) 



i=0 



where the initial and the combined state transition and observation densities are given by (|3.4I) 
and (13.51) . Assume that the true probability density of Xt+i and Yt admits a similar Markov 
structure of the form 

T 

/(Xt+1, Ft) = Uxo)X{Uzt\xt) ■ (3.8) 



By taking the expectation of 



1^ (fjXT+uYTh ^ ^^^^ ^ 

V/(XT+i,rT)^ ^0(^0)^ ^V0,( 



Zt\Xt) 



Zt\Xt) 



with respect to /(Xt+i, It), we find that the relative entropy between /(Xt+i, Yt) and /(Xt+i, It) 
satisfies the chain rule 

T 

D{f, /) = D{fo, /o) + J2 ^('^*' '^*) ' (3.9) 



t=o 



with 




M^t\xt) ft{xt) In ( ]dztdxt , 



(3.10) 
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where ft{xt) denotes the true marginal density of Xt- Up to this point, most results on robust 
Kalman filtering with a relative entropy constraint have been obtained by considering a fixed 
interval and applying a single constraint to the relative entropy D(f, f) of the true and nominal 
probability densities of the state and observation sequences over the whole interval. This was also 
the point of view adopted in Section 4 of yj which examined the robust causal Wiener filtering 
problem over a finite interval. To treat the robust filtering problem over an infinite horizon, one 
approach consists of dividing the divergence over a finite interval by the length T of the interval, 
and letting T tend to infinity, assuming this sequence has a limit. This is the case for stationary 
Gaussian processes, since in this case the limit is the Itakura-Saito spectral distortion measure 
Sec. 3]. Alternatively, it is also possible to apply a discount factor [15] to future additive terms 
appearing in the chain rule decomposition (13.91) . However, one potential weakness of applying 
a single divergence constraint to the filtering problem over a finite or infinite interval is that 
it allows the maximizer (nature) to identify the moment where the dynamic model (|3.1I) - (|3.3I) 
is most susceptible to distortions and to allocate most of the distortion budget specified by the 
tolerance parameter c to this single element of the model. If the purpose of robust filtering is to 
protect the estimator from modelling inaccuracies, and if the modeller exercises the same level of 
effort to characterize each time component of the model (I3.1I) - (|3-3I) . it may be more appropriate 
to specify separate modelling tolerances for each time step of the transition density (13.51) . Such a 
viewpoint has in fact been adopted widely [[TT ] |- lfr3ll in the robust state-space filtering literature, 
except that in these earlier studies the tolerance is usually expressed in terms of matrix bounds 
involving the matrices At, Bt, Ct and Dt parametrizing the state dynamics and observations. 
The main difference with these earlier studies is that we use here the relative entropy D{(j)t, 4>t) 
between the true and nominal transition and observation densities (l)t{zt\xt) and (j)t{zt\xt) at time 
t to measure modelling errors. 

The expression (|3.10l) for the relative entropy raises immediately the issue of how to choose 
the probability density ft{xt) used to evaluate the divergence. We assume that, like the estimating 
player, at time t the maximizer has access to the observations < s < t — 1} collected 

prior to this point. In addition, it is reasonable to hold the maximizer to the same Markov 
structure (specified by (|3.8I) ) as the estimating player. Therefore, the maximizer is required to 
commit to all the least-favorable model components (j)s{zs\xs) with < s < t — 1 generated at 
earlier stages of its minimax game with the estimating player. Using the terminology coined in 
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El, IfTSl . the maximizer operates "under commitment." Thus if i denotes the vector formed 
by the observations {y^, < s < t — 1}, we use the conditional density ft(xt\Yt_i) based on 
the least favorable model and the given observations prior to time t, to evaluate the divergence 
(13.101 ) between the true and nominal transition and observation densities. The model mismatch 
tolerance can therefore be expressed as 



E 



In 



<Ct 



^t[Zt\Xt) 

where q denotes the tolerance parameter for the time t component of the model, with 

\{zt\xt] 



(3.11) 



E 



In 



Yt- 




(l)t{zt\xt) ft{xt\Yt-i) In 



(t)t{zt\xt] 



(3.12) 



^4>t{zt\xt) 

Let Bt denote the convex ball of functions (j)t{zt\xt) satisfying inequality (13.1 II) . If Qt de- 
notes the class of estimators with finite second-order moments with respect to all densities 
<Pt{zt\xt) ftixt\Yt_i) such that (j)t{zt\xt) G Bt, the dynamic minimax game we consider can be 
expressed as 

(3.13) 



where 



9t) 



2 
1 

2 




min max Jt{(j)t,gt) 



xt+i - gtiyt)\\'^\Yt-i 

Xt+1 - gtiytWMzt\xt)ftixt\Yt^i)dztdxt . 



(3.14) 



denotes the mean-square error of estimate Xt+i = gtivt) of Xt+i evaluated with respect to the 
true probability density of Zt. Note that since Xt+i is a function of Yt, it depends not only on yt, 
but also on earlier observations, but this dependency is suppressed to simplify notations. Note 
that by taking iterated expectations, we have 



-^[1 \xt+i " Xt+i 



E[E[\\xt+^ - Xt+^\\' Yt_,]] 

JMugt)ft-i{Yt-i)dYt-i 



(3.15) 



where ft-iiXt-i) represents the least favorable density of observation vector Ff_i based on the 
least-favorable model increments selected by the maximizer up to time t — 1 . Since this density 
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is non-negative and independent of both 0^ and gt, the game (13.131 ) is equivalent to 

min max £"[1 \xt+i — ^t+il • 
gteGt ^t^Bt 



(3.16) 



This indicates that the robust least-squares filtering problem we consider is local in the sense that 
the estimator and the maximizer focus respectively on minimizing and maximizing the mean- 
square estimation error at the current time. In other words, in addition to being committed to past 
least-favorable model increments (ps with 0<s<t — lit has already selected, the maximizer 
is confined to a myopic strategy, where 0t is selected exclusively to maximize the mean-square 
estimation error at time t. By doing so, the maximizer foregoes the possibility of trading off a 
lesser increase in the mean-square error at the current time against larger increases in the future. 

If we compare the dynamic estimation game (13.131) and its static counterpart (12.51) . we see that 
the two problems are similar, but the dynamic game (|3.13l) includes a conditioning operation with 
respect to the prior state Xt, combined with an averaging operation with respect to ftixt\Yt^i). 
Thus Lemma 1 and Theorem 1 of [IJ need to be extended slightly to accommodate these 
differences. Before proceeding with this task it is worth pointing out that we do not require 
that <j)t{zt\xt) should be a conditional probability density for each Xt. It is only required that the 
product ^tizt\xt) ft{xt\Yt^i) should be a probability density for 



Zt 



Xt+l 

yt 

Xt 



so that 



A 



H\Vt 




(j)t{zt\xt)ft{xt\yt~i)dztdxt 



(3.17) 



To put it another way, the maximizer' s commitment to earlier components of the least- favorable 
model is only of an a-priori nature, since the a-posteriori marginal density of Xt specified 
by the joint density (l)t{zt\xt)ft{xt\Yt-i) is not required to coincide with the a priori density 
ft{xt\Yt-i). However, by integrating out zt, the resulting a-posteriori density will be of the form 

'ipt{.Xt)ft{xt\Yt-i) where il)t{xt) does not depend on Yt-i. 

Relation to prior work: At this point it is possible to compare precisely the robust filtering 
problem discussed here with earlier formulations of robust filtering with a relative entropy 
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tolerance considered in [[HI, [[HI, HH. In [[HI, [[HI, Hansen and Sargent introduce the 
likelihood ratio function 

between the distorted joint density of states {xg, < s < t + and observations {i/s, < s <t} 
and their nominal joint density. Setting 

/o(a;o) 

if Zt denotes the a-field generated by {zg, < s < t} and xq, we have 

E[Mt\Zt_^] = Mt.i , 

for t > 0, where E[-] denotes the expectation with respect to the nominal model distribution, so 
Mt is a martingale. Then the relative entropy between the perturbed and nominal models over 
interval < t < T can be expressed as 

D{f,f) = E[MT\nMT]. (3.19) 

Let 

T T 

J = Y,E[\\xt-Xt\\^] = J^El^tW^t-XtW^] (3.20) 

t=0 t=0 

denote the sum of the mean square estimation errors over interval [0,T], where the estimate 
Xt = gtiYt) depends causally on the past and current observations. If g = {gt,t > 0), the robust 
Kalman filtering problem of tfT4l . [fTSl and the robust causal Wiener filtering problem of [1 , Sec. 
4] can be written as 

minmaxJ (3.21) 

9 f 

where / satisfies the constraint 

£)(/, /) = E[Mt In Mt] < c . (3.22) 

Since the causal Wiener filtering problem of [1, Sec. 4] is treated as a constrained static 
estimation problem, it does not require any additional structure. On the other hand for the robust 
Kalman filtering problem of [iT4l . [|T5ll . the minimax problem (13.211) is formulated dynamically 
by introducing the martingale increments 

^ if M,_i>0 
mt={ ■ (3.23) 

= 0, 
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which can be expressed here as 



mt 



Then the chain rule (|3.10l) takes the form 



E[Mt In Mt] = ^E[Mt-iE[mtlnmt\Zt-i]] + E[M_ilnM_ 



ij ' 



and the dynamic minimax problem considered in lfT4l . IfTSl can be expressed as 



min max J 

gt,t>l mt,t>l 



(3.24) 



(3.25) 



where gt and mf > are adapted respectively to yt (the sigma field spanned by Yt) and Zf. 
Comparing (13.161) and (13.251) . we see that (13.161) represents just a local or incremental version of 
minimax problem (13.251) . In this respect it is worth pointing out that by integrating the incremental 
relative entropy constraint (13.1 II) with respect to the least favorable density ft-i{Yt-i) of Yt^i 
and summing over all < t < T, we find that cumulatively, the incremental constraints (|3.11l) 
imply 



T 



D{fJ)<}_^Ct + D{foJo), 
t=o 



(3.26) 



which has the form (13.221) . In other words, the incremental constraint (|3.1 II) just represents a 
way of dividing the relative entropy tolerance budget c in separate portions Ct allocated to the 
distortion of the Markov model transition at each time step. 

The robust filtering problem we consider can also be interpreted in terms of the robust filtering 
approach described in [fT2l . To do so, assume that the distorted transition 



At 
Ct 



is Gaussian and admits a parametrization similar to (13.51) . By using expression (12.141) for the 
relative entropy of Gaussian densities, and denoting 

^At = At-At , ACt = Ct-Ct:, 



we find that conditioned on the knowledge of xt 



D{(j)t{.\xt),(j)t{-\xt)) = - 



AAt 
ACt 



Xt\ 



k: 



1 + tr( K^^Kt - / ) - lndet{Ki^K, 



(3.27) 
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where for simplicity we fave used the compact notation Kt = K^^^^t ^^id Kt = Kz^\xf Then, if 

is Gaussian and Wt = Vt + Xtxf, the expression (13.101) for the relative entropy between the 
distorted and nominal transitions at time t yields 

AAt 



ACt 



(3.28) 



1/2 1/2 

where K^ and W^ are arbitrary matrix square roots of Kt and Wt, respectively, and \\.\\f 
denotes the Frobenius norm of a matrix [|27l p. 291]. The first term of expression (13.281) is a 
weighted matrix norm of the perturbations AAt and ACt of the state transition dynamics which 
is similar in nature to the mismodelling measures considered in [[T2l|. [[T3l . On the other hand, the 
second term models the distortion of the process and measurement noise covariance Kt = Kz^\xt, 
and is different from the distortion metrics considered in [fT2|. [fT3l . Thus the robust filtering 
problem we consider can on one hand be viewed as incremental version of the results of [14J, 
ifTSl for robust filtering with a relative entropy tolerance, but it can also be viewed as a variant 
of the robust Kalman filters discussed in [[T2l. lfT3l with a different local mismodelling measure. 

Formulation without commitment: The commitment assumption can be removed from the 
incremental formulation of robust filtering described above by adopting the conceptual framework 
of Hansen and Sargent in [[141 Chap. 18] and [[T6ll . |[28l for robust filtering without commitment. 
The key idea is, at time t, to apply distortions to both the transition dynamics (ptiztlxt) and the 
least- favorable a-priori distribution ft{xt\Yt_i). In |[T6l this is accomplished by introducing two 
distortion operators with constant risk-sensitivity parameters 6i and 02, acting respectively on 
the transition dynamics and on the a-priori density at time t based on the prior distortions and 
observations. Here, if ftixt\Yt^i) denotes a distorted version of ftixt\Yt-i), in addition to the 
constraint (13.1 II) for transition dynamics distortion, we could impose a constraint of the form 

D{ft,ft)<dt (3.29) 

on the allowed distortion of the least favorable conditional distortion for Xt based on the past 
observations Yt^i Then in the minimax problem (13.131) the maximization can be performed 
jointly over pairs ft) satisfying constraints (13.111) and (|3.29l) . Since the two constraints are 
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convex, the structure of the resuking minimax problem is similar to (I3.13I ). The main difficulty 
is algorithmic. The inner maximization introduces two Lagrange multipliers which need to be 
selected such that Karush-Kuhn-Tucker conditions are satisfied. The computation of the Lagrange 
multipliers seems rather difficult, in contrast to the case of a single Lagrange multiplier arising 
from the formulation of robust filtering with commitment. So we focus here on the case with 
commitment, leaving open the possibility that an implementable algorithm might be developed 
later for robust filtering without commitment under incremental constraints (13.1 II) and (13.291) . 
Finally, note that a third option would be to combine the distortions for the transition density 
(ptiztlxt) and for conditional density ft{xt\Yt-i) and to apply a single relative entropy constraint 
to the product distorted density (f)t{zt\xt) ft{xt\Yt-i) . Since this product corresponds to the least 
favorable joint demsity of Zt and Xt, Theorem 1 is applicable to this problem, so it is not 
necessary to analyze this version of the robust filtering problem. 

IV. Robust Minimax Filter 

The solution of the dynamic game (|3.13l) relies on extending Lemma 1 of |T| to the dynamic 
case. We start by observing that the objective function Jt{4>f,gt) specified by (|3.14l) is quadratic 
in gt, and thus convex, and linear in (pt and thus concave. The set Bt is convex and compact. 
Similarly Qt is convex. It can also be made compact by requiring that the second moment of 
estimators gt E Gt should have a fixed but large upper bound. Then by Von Neumann's minimax 
theorem [|29[ p. 319], there exists a saddle point {(p^,g^) such that 

jSu 9t) < JMl g't) < JMl 9t) ■ (4.1) 

The real challenge is, however, not to establish the existence of a saddle point, but to characterize 
it completely. The second inequality in (14.11) implies that estimator g^ is the conditional mean 
of xt+i given Yt based on the least-favorable density 

/m(x..i|r*) = /0^(^dx,)/.(x.|y,,Ocix. ^^_2) 

J j (t)^^{zt\xt)ft{,Xt\Yt^l)dxt+ldxt 
obtained by marginalization and application of Bayes' rule to the least-favorable joint density 

(i)^ti.Zt\xt)ft{.Xt\Yt-i) of {zt,xt) = (xt+uVuXt) given The robust estimator is then given by 

xt+i=9t{yt) = E[xt+i\Yt] 

= jxt+ift+i{xt+i\Yt)dxt+i. (4.3) 
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Together, the conditional density evaluation (14.21 ) and expectation (14.31 ) implement the second 
inequality of saddle point identity (14.11) . Let us turn now to the first inequality. For a fixed 
estimator g^, it requires finding the joint transition and observation density (j)'^{zt\xt) maximizing 
Jt{(pt, Qt) under the divergence constraint (13.111) . The solution of this problem takes the following 
form. 

Lemma 1: For a fixed estimator gt E Qu the function 0° maximizing Jt{<pt,gt) under constraints 
dMD) and (I3TT7]) is given by 

(ptiztlxt) = ^^1^^^ exp (^^J\xt+i - gtiyt)\f^Mzt\xt) ■ (4.4) 

In this expression, the normalizing constant Mt{\t) is selected such that (|3.17l) holds. Further- 
more, given a tolerance Q > 0, there exits a unique Lagrange multiplier A( > such that 

A(0t°,0j = Q. (4.5) 

Proof: For a given gt, the function Jt{(pt,gt) is linear in (pt and thus concave over the closed 
convex set Bt, so it admits a unique maximum in Bt. Because of the linearity of Jt with respect 
to (pt, this maximum is in fact located on the bpundary of Bt. To find the maximum, consider 
the Lagrangian 

Lt{4>t, At, fit) = Jt{4>t, g) + At(Q - Dt{4>t, <Pt)) + fit{l - iMt)) , (4.6) 

where the Lagrange multipliers A( > and fit are associated to inequality constraint (13.111) 
and equality constraint (I3.17h . respectively. We do not require explicitly that 4)t{zt\xt) should 
be nonnegative, since the form (14.41) of the maximizing solution indicates that this constraint is 
satisfied automatically. 

Then the Gateaux derivative [|30l p. 17] of Lt with respect to (pt in the direction of an arbitrary 
function u is given by 

Vt Lt{(f)t, Xt, fJ't) = lim hLt{(f)t + hu,Xt, fit) - Lt{(f)t, Xt, fit)] 

lr\\xt+i - gtiyt)\\^ - (At + fit) - At In (^) u{zt,Xt)ft{xt\Yt-i)dztdxt . (4.7) 
L2 '0t'J 

The Lagrangian is maximized by setting \/ j^Lt{(j)t, Xt, fit) = for all functions u. Assuming 
At > 0, this gives 

1^ fx) = 7^1 l^m - 9tiyt)\\' - In Mt , (4.8) 
V0f/ 2At 
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where 

lnM,^l + ^. 

At 

Exponentiating (14.81) gives (14.41) . where to ensure that normalization (I3.14h holds, we must select 

Mt{\t) = J j exp(^^J\xt+i-gt{yt)\\^^Mzt\xt)ft{xt\Yt_i)dztdxt. (4.9) 

At this point, all what is left is finding a Lagrange multiplier such that the solution 0° given 
by (14.41) satisfies the KKT condition 

Xt{ct-Dt{^'l,(f)t) = 0. 

Since we already know that the maximizing 0° is on the boundary of Bt, the Lagrange multiplier 
\t > 0, so the KKT condition reduces to (14.51) . By substituting (14.41) inside expression (13.1 II) for 
we find 

D{4>'tAt) = YM4>t,9t)-HMt{Xt)) . (4.10) 



Differentiating \nMt{Xt) gives 

'^-lnM,(AO = -^ 
Ai 

so that 



— \nMtiXt) = --^Jt{<l)'',,gt), (4.11) 
dXi Xf 



7(At) = D{<j)lpMt) = -Xt^\nMt{Xt)-\n{Mt{Xt) . (4.12) 

a At 



The derivative of 7(At) is given by 



= -At— 2 In Mi -2— In Mi 
aXf dXr dXt 



1 

x 

1 



d d Y M 
dXt V * dXt * 



1 d 



Xt dXt 

3E[(||xt+i-(7t(yt)||2-E[||xt+i-(7t(yt)|nFt_i])Vt_i] <0, (4.13) 



4At 

so that 7(At) is a monotone decreasing function of A^ As At — )■ oo we have obviously 0° — t- 0t, 
so that 7(00) = 0. Thus provided q is located in the range of 7t, which is the case if q is 
sufficiently small, there exists a unique Cf such that 7t(At) = Q. □ 

Note that Lemma 1 makes no assumption about the form of the nominal transition density 
0t(-2t|a;t) and a priori density /t(a;t|l^t_i). Without additional assumptions, it is difficult to 
characterize precisely the range of function 7^. When both of these densities are Gaussian, 
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it will be shown below that the range of 7^ is IR+, so that any positive divergence tolerance 
Ct can be achieved. However, in practice the tolerance Ct needs to be rather small in order to 
ensure that the robust estimator is not overly conservative. At this point is is also worth observing 
that Lemma 1 is just a variation of Theorem 2.1 in llBTl p. 38] which sought to construct the 
minimum discrimination density (i.e., the density minimizing the divergence) with respect to 
a nominal density under various moment constraints. Here we seek to maximize the moment 
£^[11x4+1 — gt{yt)\\'^\Yt-i] under a divergence constraint. From an optimization point of view, the 
two problems are obviously similar, and in fact the functional form (|4.4I) of the solution is the 
same for both problems. 

Up to this point we have made no assumption on either the nominal transition and observations 
density (f)t{zt\xt) and estimator gt, and in the characterization of the saddle point solution (0°, g^) 
provided by identities (|4.3I) and (14.41) . the robust estimator g'^ depends on least-favorable transition 
function 0°, and the least favorable transition density (f)^ depends on robust estimator g^. This type 
of deadlock is typical of saddle point analyses, and to break it, we introduce now the assumption 
that (f)t{zt\xt) admits the Gaussian form (13.51) where as indicated earlier, the covariance matrix 
Kzt\xt is positive definite, and we assume also that at time t the a-priori conditional density 

ft{xt\Yt-i) ^{xt,Vt) . (4.14) 

Then, observe that the distortion term exp(||a;t+i — gt{yt)\\'^ / {2Xt)) appearing in expression (|4.4I) 
for the least-favorable transition function (j)^(zt\xt) depends only on zt, but not xt. Accordingly, 
if we introduce the marginal densities 

ft{zt\Yt-i) = j (l)t{zt\xt)ft{xt\Yt-i)dxt (4.15) 
/°(zt|y;_i) = j 4>K^t\xt)ft{xt\Y,_,)dxt, (4.16) 

the density f{zt\Yt-i) can be viewed as the pseudo-nominal density of zt = {xt+i, yt) conditioned 
on Yt^i computed from the conditional least favorable density ftixt\Yt_i) and nominal transition 
density and from (|4.4I) we obtain 

fti^t\Yt-i) = j^^^P [^J\xt+, - gtiytW) ftiztlY,^^) . (4.17) 

Since densities (f)tizt\xt) and ft{xt\Yt-i) are both Gaussian, the integration (14.151) yields a 
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Gaussian pseudo-nominal density 



At 
Ct 



^ti Kz 



(4.18) 



where the conditional covariance matrix is given by 

At 
Ct 



Vt 







Bt 






'_AJ Cf_ 


+ 




\bI of 




Dt 





(4.19) 



By integrating out Xt in (I4.9h . we find 

Mt{\t) = J exp - 9oiyt)\\^^ftizt\Yt-i)dzt , 

which ensures that f^{zt\Yt^i) is a probability density. Furthermore, by direct substitution, we 



have 



D{f^Jt) = D{^l<Pt) = Ct. 



(4.20) 



The least-favorable density ft+i{xt+i\Yt) specified by (14.21) can also be expressed in terms of 

ff{zt\Yt-i) as 



ft+i{xt+i\Yt) 



mzt\Yt-,] 



J f,'{zt\Yt^i)dxt+i 



(4.21) 



Equivalent Static Problem: Let 



Bt = {ft:Dift,ft)<Ct} 



(4.22) 



denote the ball of distorted densities ftizt) within a divergence tolerance q of pseudo-nominal 
density ft{zt\Yt-i). To this ball we can of course attach a static minimax estimation problem 



min maxJt{ft,gt) ■ 
gt&Gt ft(^Bt 

The solution of this problem satisfes the saddle point inequality 

Jifl9^)<Jifl,9^)<Mfl,9t). 



(4.23) 



(4.24) 



At this point, observe that if (0°, g^) solves the dynamic minimax game (13.121) . and if /° is given 
by (14.171 ) with gt = g^, where is selected such that constraint (14.201) is satisfied, then (f^,g^) 
is a saddle point of the static problem (14.231) . In other words, the marginalization operation 
(14.161) has the effect of mapping the solution of dynamic game (13.121) into a solution of the 
static estimation problem problem (14.231 ). Note indeed that the solution of the maximization 
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problem formed by the first inequality of (14.241) is given by (14.171) with gt = Qt- Similarly, since 
is the mean of the conditional density ft+i{xt+i\Yt) specified by (|4.21l) . it obeys the second 
inequality of (14.241 ). 

Since the pseudo-nominal density ft{zt\Yt-i) specifying the center of ball Bt is Gaussian, 
Theorem 1 is applicable with f ^ ft, fo^ ft and go ^ g^- Hence the least-favorable density 
takes the form 

'At 



f^{zt\Yt.r)^N 



Ct 



Xt,K;, 



where the covariance matrix 



K 



2t 



K K 
K K 



yt 



is obtained by perturbing only the (1,1) block 

K,^^, = AtVtAj + BtB] 



of the covariance matrix K^^ given by (|4.19l) . The robust estimator takes the form 

S^t+i = gtivt) = AtXt + Gt{yt - Ctxt) 

with the matrix gain 



Gt = K^^^.y^K-^' = {AtVtCj + BtDj){CtVtCj + DtDjy' 
The least-favorable covariance matrix Kxt+i ^^''^ be evaluated as follows. Let 



Pt+i 



^^Xt+i Xt+iyt-^^ yt ^^VtXt+i 



{At - GtCt)Vt{At - GtGtf + {Bt - GtDt){Bt - GtDt 



and 



(4.25) 



(4.26) 



(4.27) 



(4.28) 



Vt+i = K.,^, - K,,^,y,K-^Ky,,,^, (4.29) 
denote the nominal and least-favorable conditional covariance matrices of Xt+i given yt- Then 



where the Lagrange multiplier > r(P(_|_i) is selected such that 

1 



tr ((/„ - X;'Pt+i)-' - In) + lndet(/„ - X^'Pt+i] 



Ct ■ 



(4.30) 



(4.31) 
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where as indicated in (12.171) . 7t(At) is monotone decreasing over (r(P(_,_i), oo) and has for range 
M+. Thus for any divergence tolerance Ct > 0, there exists a matching Lagrange multiplier 

Summary: The least-favorable conditional distribution of Xt+i given Yt is given by 

~ft+i{xt+i\Yt) ~ U{xt+i, Vt+i) , (4.32) 

where the estimate Xt+i is obtained by propagating the filter (I4.26l) - (|4-27l) and the conditional 
covariance matrix Vt+i is obtained from (|4.28l) and (14.301) . with the Lagrange multiplier A( 
specified by (|4.31|) . By writing 9t = A^^, we recognize immediately that the robust filter is a form 
of risk- sensitive filter of the type discussed in ^ [|32l Chap. 10]. However, there is a new twist 
in the sense that, whereas standard risk-sensitive filtering uses a fixed risk sensitivity parameter 
6, here 6 is time-dependent. Specifically, in classical risk-sensitive filters, 6 is an exponentiation 
parameter appearing in the exponential of quadratic cost to be minimized. Similarly, in earlier 
works [fT4|. ifTSl . ifTTl . ifTSl relating risk sensitive filtering with minimax filtering with 
a relative entropy constraint, a single global relative entropy constraint is imposed, resulting 
in a single Lagrange multiplier/risk sensitivity parameter. Here each component (j)t{zt\xt) of the 
model has an associated relative entropy constraint (|3.10l) . where the tolerance q varies in inverse 
proportion with the modeller's confidence in the model component. In this respect, even if the 
state-space model (I3.1I) - (I3.2I) is time-invariant {A, B, C and D are constant) and the tolerance 
Q = c is constant, the risk sensitivity parameter A^^ will be generally time-varying. On the other 
hand, if we insist on holding 9 = A^^ constant, it means that the tolerance q = 7f(A) is time 
varying since the covariance matrix Pt^i given by (14.281) depends on time. 

V. Least-Favorable Model 

In last section, we derived the robust filter, which is of course the most important component 
of the solution of the minimax filtering problem. However for simulation and performance 
evaluation purposes, it is also useful to construct the least favorable model corresponding to 
the optimum filter. Before proceeding, note that if et = Xt — Xt denotes the state estimation error, 
by subtracting (14.261) from the state dynamics (13.11) and taking into account expression (|3.2|) for 
the observations, the estimation error dynamics are given by 

Ci+i = {At - GtCt)et + {Bt - GtDt)vt , (5.1) 
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Vt = i-t \Zt 



xt . (5.3) 



where in the nominal model, the driving noise Vt is independent of error et = Xt — Xt, since Xt 
depends exclusively on observations {y{s), < s < t — 1}. 

To find the least-favorable model, we use the characterization (14.41) where gt = g\ is given by 
the robust filter ( 14.261 ). This gives 

1 II IP 

At this point, recall that ^\[^Zt\xi) is an unormalized density. Specifically, integrating it over 
zt does not yield one, but as we shall see below, a positive function of — Xt — Xt. This 
feature indicates that the maximizing player has the opportunity to change retroactively the 
least-favorable density of x^ (and therefore of earlier states) by selecting the model component 
0°(2(|xt). Properly accounting for this retroactive change forms an important aspect of the 
derivation of the least-favorable model. Instead of attempting to characterize directly the least- 
favorable density of Zt given Xj, it is easier to find the least-favorable density of the driving 
noise 

'a 

Ct 

Given xu the transformation (15.31) establishes a one-to-one correspondence between zt and vu 
so there is no loss of information in characterizing the least-favorable model in terms of Vt- 
Let i^tivt) and V^t(ft|et) denote respectively the nominal and least-favorable densities of noise 
Vt, where as will be shown below, V^t(ft|et) actually depends on et. The nominal distribution is 
given by 

Mvt) = j^^eM~\\vt\\V2). (5.4) 

Assume that we seek to construct the least-favorable model of Zt over a fixed interval < t < 
T, and that the least-favorable noise distribution ^sivs\es) has been identified for t + l < s <T. 
Accordingly, we have 

T II ||2 T 

s=t+l * s=t+l 

where ~ indicates equality up to a multiplicative constant. The term exp (| |et+i| /2) appear- 
ing in the above expression accounts for the cumulative effect of retroactive probability density 
changes performed by the maximizing player. Here Vlt denotes a positive definite matrix of 
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dimension n which is evaluated recursively. Then the least-favorable model ipt{vt\et) is obtained 
by backward induction. Decrementing the index t by 1 in (15.51) gives the identity 

exp (^(||et+i||^-i^ + A7^||et+i||^)/2j?/'i(ft) ~ ?/'t(t;i|ef) exp (| |et| |^-i/2) . (5.6) 

Let 

Wt+, = {Q;^, + X;'lnr' . (5.7) 
Then by substituting the error dynamics (15. 1|) . the left hand side of identity (15. 6|) becomes 

exp - GtQ)et + {B^ - G^A)^*)!!'^-! - \\vt\\')/2) , (5.8) 

and the right-hand side of (15.61) is obtained by decomposing the quadratic exponent of (|5.8I) as a 
sum of squares in Vt and e^. By doing so, we find that the least-favorable noise density is given 
by 

Mvt\et)--Af{Htet,K,), (5.9) 

where 

K = [in+p - [Bt - GtDtYW-^^{Bt - GtDt)) ' (5.10) 

and 

= K,XBt - GtDtfW,-\{At - GtGt) . (5.11) 

Thus the least-favorable density of the noise Vt involves a perturbation of both the mean and 
the variance of the nominal noise distribution. The mean perturbation is proportional to the 
filtering error et, which creates a coupling beetween the robust filter and the least favorable 
model specified by dynamics and observations (I3.1I) - (I3.2I) and least-favorable noise statistics 

Finally, by matching quadratic components in et on both sides of (15.61) . we find 

Q;' = (A - GtGtfW,-\{At - GtGt) - Hjk.,^Ht (5.12) 

where Ky^ and Ht are given by (|5.10l) and (15.1 II) . By using the matrix inversion lemma [[33l p. 
48] 
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with a = Wt, l3 = [Bt — GtDt), 7 = —In+p and 5 = {Bt — GtDtY on the right-hand side of 
(|5.12l) . we obtain 

= {At - GtCtf{Wt+i - {Bt - GtDt){Bt - GtDtf]-\At - GtGt) . (5.13) 

The recursion (I5.13L together with (15.71 ) specifies a backward backward recursion which is used 
to account for retroactive changes of previous least-favorable model densities performed by the 
maximizing player. The backards recursion is initialized with VLj.]^^ = 0, or equivalently, 

W^T+i = At/„+p . (5.14) 

In this respect, it is interesting to note that recursion (15.131 ) can be rewritten in the forward 
direction as 

Wt+i = {At - GtGtWAt - GtGtf + {Bt - GtDt){Bt - GtDtf (5.15) 

and (15.71) is of course equivalent to 

nt+i = {Wt-^\-X;'ln)-' . (5.16) 

Thus rit and Wt obey exactly the same forward recursions as as Vt and Pt, but they are computed 
in the backward direction. Indeed, observe that the matrices and Wf^ are typically very 
small, so it is much easier to maintain positive-definiteness by using (15.71) which accumulates 
small positive terms, instead of using (15.161) which subtracts a small positive-definite matrix from 
another one. 

The least-favorable noise model (I5.9I )- (I5.1 II) can be viewed as a time-varying version of the 
least-favorable model derived asymptotically for the case of a constant model by Hansen and 
Sargent in [[T4l Sec. 17.7]. Specifically, the dynamics of the least-favorable model described in 
lfT4l are expressed in terms of the solution of a deterministic infinite-horizon linear-quadratic 
regulator problem. The counterpart of this regulator is formed here by backward recursion (15.131) . 
(I5J1) . 

The model (I5.9I )- (I5.11I) indicates that the driving noise Vt admits the representation 



Vt = Htet + Ltet (5.17) 
where Lt is an arbitrary matrix square root of Ky^, i.e., 

LtLj = [In+p - {Bt - GtDtf Wt-^\{Bt - GtDt)]-' , (5.18) 
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and ej is a zero-mean WGN of variance In+p- Accordingly, as was previously observed in [14], 
if 



6 



A 



Xt 

et 



(5.19) 



the least-favorable model admits a state- space representation 

^t+i = Atit + Btet 
yt = Ctit + Dtet 
with twice the dimension of the nominal state-space model, where 



(5.20) 



At 



At BtHt 
At-GtCt + {Bt-GtDt)Ht 



Dt 



(Bt 
DtLf 



Bt 

- GtDt) 



(5.21) 



Ct DtHt 

Note that the model (|5.20I) - (I5.21I) is constructed by performing first a forward sweep of the 
risk-sensitive filter (|4.27| )- (|4.28l) over interval [0,T] to generate the gains Gt, followed by a 
backward sweep used to evaluate the matrix sequence Wt- Thus, the least-favorable model is 
constructed in a nonsequential manner, since increasing the simulation interval beyond [0,T] 
requires performing a new backward sweep of recursion (15.131 ), (15.71 ). 

The model (15.201) can be used to assess the performance of any estimation filter designed under 
the assumption that the nominal model (|3.1|) - (I3.2|) is valid. Let G[ be an arbitrary time-dependent 
gain sequence, and let x[ be the state estimate generated by the recursion 



x'l = ^tx't + G[{yt - Gtx\) . 



(5.22) 



Let = — x\ denote the corresponding filtering error. When the actual data is generated by 
the least-favorable model (|5.20I) - (I5.21| ), by subtracting recursion (15.221) from the first component 
of the state dynamics (15.201) . we obtain 

. N r 1 / . \ G'.'\ ^ ^ 

BtVt. 







G't 






+ {Bt- 


G't 


et+1 









et 








(5.23) 

The recursion (15.231 ) can be used to evaluate the performance of filter (15.221 ) when the data is 
generated by the least- favorable model (I5.20I) - (I5.21|) . Specifically, consider the covariance matrix 



Ilt = E 



et 
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By using the dynamics (15.231 ) derived under the assumption that the data is generated by the 
least-favorable model, we obtain the Lyapunov equation 



t+i 



A, 







cAuAAt- 



G' 



+ [Bt- 



G't 




G't 




(5.24) 



which can be used to evaluate the performance of filter (15.231) when it is applied to the least- 
favorable model. For the special case where G\ is the Kalman gain sequence, this yields the 
performance of the standard Kalman filter. 

VI. Simulations 

To illustrate the behavior of the robust filtering algorithm specified by (I4.26I) - (I4.31I) . we 
consider a constant state- space model employed earlier in [[TT]|. [[T2l: 



A 



0.9802 0.0196 
0.9802 

C = 



BB 



T 



DD 



T 



Q 



1 . 



1.9608 0.0195 
0.0195 1.9605 



The nominal process noise Bvt and measurement noise Dvt are assumed to be uncorrelated, so 
that BD^ = 0, and the initial value of the least-favorable error covariance matrix is selected as 

We apply the robust filtering algorithm over an interval of length T = 200 for progressively 
tighter values 10^^, 10^^ and 10^^ of the relative entropy tolerance c. The corresponding time- 
varying risk-sensitivity parameters 9t = obtained from (14.311) are plotted in Fig. \T\ The 
least-favorable variances (the (1,1) and (2,2) entries of Vt) of the two states are plotted as 
functions of time in Fig. [2] and Fig. [3l As can be seen from the plots, although the relative 
entropy tolerance bounds that we consider are small, increasing the tolerance c by a factor 10 
leads to an increase of about 7dB in the state error variances. 

Next, for a tolerance c = 10~^, we compare the performance of the risk-sensitive and Kalman 
filters for the nominal model, and for the least-favorable model constructed as indicated in 
Section |Vl The variances of the two-states for the nominal model are shown in Fig. |4] and 
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Fig. 1. Plot of time varying parameter 9t — X^^ (logarithmic scale) for c = 10 ^,10 ^ and 10 

Fig. m respectively. Clearly, the loss of performance of the risk-sensitive filter compared to the 
Kalman filter is less than IdB. 

On the other hand, as indicated in Fig. [6] and Fig.|7l when the risk-sensitive and Kalman filters 
are applied to the least-favorable model, the Kalman filter performance is about 8dB worse than 
the robust filter. Note that to allow the backward recursion (15.131 ). (15.71 ) to reach steady state, the 
backward model is computed for a larger interval, and only the first 200 samples of the simulation 
interval are retained, since later samples are affected by transients of the least-favorable model. 

VII. Conclusion 

In this paper, we have considered a robust state-space filtering problem with an incremental 
relative entropy constraint. The problem was formulated as a dynamic minimax game, and 
by extending results presented in |[T1, it was shown that the minimax filter is a risk-sensitive 
filter with a time varying risk-sensitive parameter. The associated least-favorable model was 
constructed by performing a backward recursion which keeps track of retroactive probability 
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Fig. 2. Error variance of xu (dB scale) for c — 10 ^,10 and 10 *. 



changes made by the maximizing player. The results obtained are similar in nature to those 
derived by Hansen and Sargent [[T4|. [fTSl for the minimax problem (13.251) when a single relative 
entropy constraint is applied to the overall state-space model and the maximizing agent is required 
to operate under commitment.. 

A number of issues remain to be resolved. For the case of a constant state-space model, it would 
be of interest to establish the convergence under appropriate conditions of the robust filtering 
recursions and of the backwards least- favorable model recursions. One also has to wonder if the 
results derived here for Gauss-Markov models could be extended to classes of systems, such 
as partially observed Markov chains, for which robust filtering with an overall relative entropy 
constraint was considered previously in [,34,1 . 
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